Optical fiber communications is seen as one of the most reliable telecommunication technologies to achieve consumers' needs for present and future applications. It is reliable in handling and transmitting data through hundreds of kilometers with an acceptable bit error rate and today, optical fiber communication dominates as the physical medium for medium and long distance data transmission systems and telecommunications networks. At the same time optical fiber solutions now appear in short-haul applications, local area networks, fiber-to-the-home/curb/cabinet, and digital cable systems. Over the same 30 year time period (1984-2014) as optical networks have evolved from initial 140 Mb/s links to wavelength division multiplexed Tb/s links microprocessors have evolved from single core 20 MHz processors to 4 and 6 core 2-4 GHz desktop and server processors and 60 core 1 GHz server processors. Meanwhile Internet evolved from a few million users on desktop computers to nearly three billion users representing approximately 40% of the global population on a range of devices from laptops through smart televisions to gaming consoles and smart phones.
Data centres are facilities that store and distribute the data on the Internet. With an estimated 14 trillion web pages on over 750 million websites, data centres contain a lot of data. Further, with almost three billion Internet users accessing these websites, including a growing amount of high bandwidth video, there is a massive amount of data being uploaded and downloaded every second on the Internet. At present the compound annual growth rate (CAGR) for global IP traffic between users is between 40% based upon Cisco's analysis (see http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/white_paper c11-481360_ns827_Networking_Solutions_White_Paper.html) and 50% based upon the University of Minnesota's Minnesota Internet Traffic Studies (MINTS) analysis. By 2016 this user traffic is expected to exceed 100 exabytes per month, or over 42,000 gigabytes per second. However, peak demand will be considerably higher with projections of over 600 million users streaming Internet high-definition video simultaneously at peak times. All of this data flowing into and out of these data centres will generally be the result of data transfers between data centres and within data centres so that these overall IP traffic flows must, in reality, be multiplied many times to establish the total IP traffic flows.
Data centres are filled with tall racks of electronics surrounded by cable racks where data is typically stored on big, fast hard drives. Servers are computers that take requests and move the data using fast switches to access the right hard drives and either write or read the data to the hard drives. In mid-2013 Microsoft stated it had itself over 1 million servers. Connected to these servers are routers that connect the servers to the Internet and therein the user and/or other data centres.
According to Facebook™, see for example Farrington et al. in “Facebook's Data Centre Network Architecture” (IEEE Optical Interconnects Conference, 2013 available at http://nathanfarrington.com/presentations/facebook-optics-oida13-slides.pptx), there can be as high as a 1000:1 ratio between intra-data centre traffic to external traffic over the Internet based on a single simple request. Within data centre's 90% of the traffic inside data centres is intra-cluster.
At the same time as requiring an effective yet scalable way of interconnecting data centres and warehouse scale computers (WSCs), both internally and to each other, operators must provide a significant portion of data centre and WSC applications free of charge to users and consumers, e.g. Internet browsing, searching, etc. Accordingly, data centre operators must meet exponentially increasing demands for bandwidth without dramatically increasing the cost and power of the infrastructure. At the same time consumers' expectations of download/upload speeds and latency in accessing content provide additional pressure.
Historically microprocessor improvements from 1984-2004 were driven through increasing clock speeds as processor speeds increased from 20 MHz to 3 GHz. Subsequently processor speeds have typically maintained in the 2.5-4 GHz range and many microprocessor manufacturers have stated that circuit speeds are unlikely to exceed 5 GHz as both static and dynamic power dissipation considerably increase for deep sub-100 nm CMOS. Already, an Intel™ Core™ i7-5960X desktop processor with 8 cores operating up to 3.5 GHz with 20 MB cache consumes up to 140 W and an Intel™ Xeon Phi™ 7120X server coprocessor with 61 cores operating up to 1.2 GHz with 16 GB cache memory consumes 300 W. Such multi-core processors have therefore driven performance enhancements of the period 2004-2104. However, in many-core architectures, the overall performance of the computing system depends not only on the capabilities of the processing nodes but also the electrical interconnection networks carrying the communications between processors and between processors and memories.
Already optical interconnection solutions play critical roles in data centre operations for the interconnection of servers, hard drives, routers etc., where the goal is to move data as fast as possible with the lowest latency, the lowest cost and the smallest space consumption on the server blade. Gigabit Ethernet is too slow and 10 Gb/s solutions such as 10G Ethernet and Fibre Channel are deployed whilst 10/20 Gb/s Fibre Channel and 40G/100G Ethernet are emerging based upon multiple 10 Gb/s channels run over parallel multimode optical fiber cables or wavelength division multiplexed (WDM) onto a singlemode fiber. Intra-rack and local inter-server communications typically exploit 100GBASE-SR10 links with OM3/OM4 multimode optical fibers providing 100 m/150 m reach. General inter-server communications within a data centre that can be a few thousand meters and hence 100GBASE-LR4 singlemode optical fiber links with reach up to 10 km may be employed. Today, in addition to addressing such link speed enhancements, focus is being made to the architectures employed within the data centre in order to reduce latency and ease physical implementation where tens of thousands of fiber optic cables may be run within the data centre. Today the largest data centres comprise 50,000 to 100,000 servers.
However, within the server the electrical interconnection networks also suffer issues when scaling to a large number of processors due to the server level interconnections albeit differing in several aspects. Simple topologies, such as a chip-global bus, exhibit high latency, require power-hungry repeaters, and occupy large footprint. More complex topologies can be exploited, such as direct networks for example, which connect neighbouring processing nodes within a predetermined topology through point-to-point dedicated links. Still, these networks just like the spline-leaf networks connecting servers require the signal to cross multiple hops for connecting distant cores and are prone to contention between concurrent message transmissions, both leading to increased latency and power consumption. Accordingly, providing additional bandwidth for inter-circuit, intra-board, and inter-board applications just as with server connections will require the adoption of optical communication solutions. Accordingly, these will require the provisioning of low cost, small footprint, and low power solutions in order to meet the requirements of the applications and ongoing market drivers. Accordingly integrated optoelectronic solutions offer a technology option addressing these requirements.
Within the prior art, optical solutions to address and overcome the issues of superseding/replacing electrical interconnection networks have generally exploited some form of optical space switching. Such optical space switching architectures required multiple switching elements, leading to increased power consumption and footprint issues. Accordingly, it would be beneficial for new optical, e.g. fiber optic or integrated optical, interconnection architectures to address the traditional hierarchal time-division multiplexed (TDM) space based routing and interconnection to provide reduced latency, increased flexibility, lower cost, and lower power consumption.
In order to address this, the inventors exploit multiple domains by overlaying mode division multiplexing to provide increased throughput in bus, point-to-point networks, and multi-cast networks, for example, discretely or in combination with wavelength division multiplexing. Further, routing within networks according to embodiments of the invention may be based upon space switching, wavelength domain switching, and mode division switching or combinations thereof. In this manner the inventors provide interconnections exploiting N×W×M×D Gb/s photonic interconnects wherein N channels are provided each carrying W wavelength division signals with M modes each at D Gb/s.
Other aspects and features of the present invention will become apparent to those ordinarily skilled in the art upon review of the following description of specific embodiments of the invention in conjunction with the accompanying figures.